Fix segfault on Windows when stdout is not a real console handle#125
Open
davesnx wants to merge 1 commit intoocaml-community:masterfrom
Open
Fix segfault on Windows when stdout is not a real console handle#125davesnx wants to merge 1 commit intoocaml-community:masterfrom
davesnx wants to merge 1 commit intoocaml-community:masterfrom
Conversation
On some Windows environments (e.g. GitHub Actions runners), _isatty() returns true for handles that are not real console screen buffers (likely due to ConPTY). Passing these handles to GetConsoleScreenBufferInfo causes an access violation (segfault). This adds two layers of defense: 1. C stub level: validate handles with GetConsoleMode before calling GetConsoleScreenBufferInfo/SetConsoleWindowInfo. Invalid handles now raise a Unix_error instead of crashing. 2. OCaml level: wrap the get_size_from_fd call in LTerm.create with a try/catch. If getting the terminal size fails, fall back to treating the terminal as non-tty instead of crashing. Fixes ocaml-community#124
davesnx
added a commit
to reasonml/reason
that referenced
this pull request
Mar 4, 2026
- Disable rtopIntegration test on Windows due to lambda-term segfault when stdout is not a real console handle (ConPTY issue on GH Actions). See: ocaml-community/lambda-term#125 - Add setup-mingw step in esy-ci for Windows to provide x86_64-w64-mingw32-gcc needed to compile lambda-term C stubs. - Bump esy cache keys to v0.0.2 to avoid stale caches.
davesnx
added a commit
to reasonml/reason
that referenced
this pull request
Mar 4, 2026
Pin lambda-term to davesnx/lambda-term#fix-windows-console-segfault in opam-ci on Windows to test the ConPTY segfault fix end-to-end. Re-enable rtopIntegration test on Windows for OCaml >= 5.0 (opam-ci with 5.4.0 will exercise the fix). esy-ci uses 4.14 so the test is naturally skipped there. See: ocaml-community/lambda-term#125
davesnx
added a commit
to reasonml/reason
that referenced
this pull request
Mar 5, 2026
* test(rtop): capture stderr in integration test for Windows * ci: setup tmate * test(rtop): avoid running rtop in windows 4.14.* * ci: enable 5.4.0 in windows * ci: remove action-tmate * ci: revert all changes, keep building * ci: add -preasonrtop in esy-ci * ci: update cache keys * ci: fix Windows CI failures - Disable rtopIntegration test on Windows due to lambda-term segfault when stdout is not a real console handle (ConPTY issue on GH Actions). See: ocaml-community/lambda-term#125 - Add setup-mingw step in esy-ci for Windows to provide x86_64-w64-mingw32-gcc needed to compile lambda-term C stubs. - Bump esy cache keys to v0.0.2 to avoid stale caches. * ci: replace setup-mingw action with direct choco install The egor-tensin/setup-mingw@v2 action fails on MinGW 15.2.0 because its cleanup script tries to remove libpthread.dll.a which no longer exists in UCRT-based builds. Use choco directly and add the bin directory to PATH manually. * ci: esy-ci doesn't run rtop tests on windows
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #124 —
utop(and anything built on it, like Reason'srtop) segfaults on Windows in certain environments (e.g. GitHub Actions runners).Problem
On some Windows environments,
_isatty()returnstruefor handles that are not real console screen buffers. This likely happens due to ConPTY (Windows Pseudo Console). When lambda-term callsGetConsoleScreenBufferInfoon such a handle, it causes an access violation (segfault) instead of returning an error.The crash path is:
LTerm.create→Lwt_unix.isattyreturnstruefor stdoutoutgoing_is_a_tty = true,get_size_from_fdis calledlt_term_get_size_from_fdcallsGetConsoleScreenBufferInfo(Handle_val(fd), &info)with no handle validationFix
C stub level (
lTerm_term_stubs.c): Validate handles withGetConsoleModebefore callingGetConsoleScreenBufferInfo/SetConsoleWindowInfo. Invalid handles now raise aUnix_errorinstead of crashing.OCaml level (
lTerm.ml): Wrap theget_size_from_fdcall inLTerm.createwith a try/catch. If getting the terminal size fails, fall back to treating the terminal as non-tty instead of crashing.Testing
Tested by reproducing the issue on a GitHub Actions
windows-latestrunner with:Before fix:
echo "1 + 1;;" | utop 2>&1→ Segmentation faultAfter fix: should raise a catchable OCaml exception and fall back to non-tty mode